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ABSIBACT 

Probles*-solving skills are higher level cognitive 
processes which involve the invention^ of cbsplez organizational 
scheses to resolve conflicts or confasion* These skills cannot be 
acquired bj mastering a hierarchy of preteguisite skills; . rather, 
their sastery is best facilitated by placing the student in an 
environment which forces his to apply thfi skill. Cospater sisalations 
can provide rich stisalation and can incorporate cosplez 
interrelationships between variables while avoiding the asbient 
distractions and'^over^-cosplezity of real-life situations. Siaulations 
can force students to: (1) identify relevant issues; (2) detersine 
sequences; (3) collect and interpret ° data; (4) avoid unnecessary 
actions; (5) sanipulate situations; and (6) evaluate the effects of 
those sanipulations. (EBB) 
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In the professions,, notably medicine, law, and education, an 
instructional goal of primary importance is the development of students' 
abilities to identify, analyze, and solve problems, applying substan- 
tive knowledge in the process. Tea'&hing and eliciting such problem- 
solving skills is accompanied by the need to evaluate the problem- 
solving skills of the would-be doctor, lawyer, or educator. 

A variety of strategies has been used to teach problem- solving 
•kills. The least costly has usually been demonstrated to be also 
iMSt effective. In legal education, for example, the predominant 
technique used to aid beginning law students in developing these skills 
is the case or Socratic mett^od. Students are expected to analyze a 
case and respond wheh called upon .in class, in a Socratic dialogue 

r 

I ' I 

with the law professor. The classes, however, typically have between 
seventy- five ^nd one-hundred students, severely limiting the amount of 
interaction an individual st,udent can expect to have with the professor. 
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A more successful approach has been the "clinic" or "internship", 
which places a student in a real-world situation whera she or he can 
observe practicing professionals solving problems , and can participate 
in the process, under supervision. Clearly, the cost of such instruc- 
tion is very high, even prohibitive. 

Evaluation of problem- solving skills has proven to be equally 
difficult. Traditional paper- and- pencil objective tests are more 
appropriate for measuring lower*level cognitive skills, it is a con- 
•iderable challenge to evaluate clinical higher-level cognitive skills 
of the type required for interaction with a client (e.g., a patient, 
a person in need of legal assistance, or a child with a learning 
deficiency) . 

The Committee on Examinations of the American Academy of Ortho- 
pedic Surgery abandoned paper-pencil Board Examinations in favor of 
a more valid and relevant evaluative interview of each candidate by a 
review board of physicians. The cost of administering such an exam- 
inatibn is prohibitively high. A reasonable com^omise was found in 
the form of a set of paper-pencil simulations called Patient Manage- 
Mnt Problems (McGuire, undated). i 

The use of a simulation to teach or evaluate problem- solving 

■ 

•kills has found many applications in medical education and the train- 
ing of teachers, as well as experimental application in legal education 
(Edwards, 1975). « 

^ ^ ..1 
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The Nature of Problem- Solving as an Intellectual Skill 

\ 

* Problem* solving appears to be an intellectual skill which is 

not simply an extension of lower^-or^cr skills such as "concept learn* 

I 

ing** or "rule learning." In fact, it appears to be qualitatively 
different from these less complex kinds of intellectual processing. 

The variation in the terminology used when discussing higher^ 
level cognitive processes reflects the state of what is currently 
known. Terms such as "problem solving," or "heuristic," are some- 

♦ 

times used interchangeably. However, the terms may have quite diff- 
erent meanings to some investigators, fuzzily defined and difficult 
to communicate as those ipeanings may be. i 

Solving a complex problem in law, for example, requires, first, 
a substantive knowledge of legal procedure, fact, and precedent. A 
lawyer in practice must al^o possess and apply skills in analyzing a 
case to identify the relevant and important issues, and must then^be 
able to apply the knowledge to resolve the case as successfully and 



•fficiently as possible, 
of skills and knowledge. 



Clearly, this requires a complex interaction 



For the purposes of this paper, the cognitive 
processes a student engages in while developing and practicing these 
■kills in a simulated environment is called "problem solving." 

Gagne (1970) identifies a hierarchy of intellectual skills, 
classified by leve^ of complexity. I At each level of the hierarchy, 
<^the simpler skills are prerequisite conditions for the more complex 
intelleti^tual skills. Most instructional design models specify a task 
analysis based on Gaa^e's hierarchy to identify the level of skill 
involved, followed ty the planning of a sequente of learning events 
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external to the learner. These instructional events will Jead to the 
acquisition of the prerequisite skills lower in the hierarchy^ which 
then act as ••sufficient conditions'* which insure the acquisition of 
the identified higher- level skill. 

Gagne & Briggs (1974) suggest that problem- solving skills (cog- 
nitive strategies) are internally organized skills which manage or 
guide the individual's learning, remembering and thinking behavior. 
Because they are internally organized, cognitive strategies cannot 
be directly controlled by external "sufficient conditions," in the 

.:\ ' . ■ 

way that lower- level skills can. Rather, the designer plans "favor- 

I 

able conditions" to incree.se the probability of certain internal 
events which in turn determine tl^ learning of the cognitive strategy. 

i 

"Favorable conditions," according co Gagne and Briggs, are those which 
provide opportunities to thinkr-to develop and use cognitive strategies 

Gagne (1970) and Gagne & Briggs (1974) characterize problem- 
solving behavior as the invention of complex combinations of rules 
to solve a problem or a class of problems. For learning to be ^ 
* problem- solving event" they would require the learning guidance to 
bo provided by the problem sblver himself, rather than the teacher 
or dther external source. The learner thus engages in ""discovery 
learning." Once acquired, the higher-order rule should exhibit 
transfer to other physically different, but formally similar, situa- 
tions. This writers suggest that; 

In attaining a workable solution to a problem, 
the student also achieves a new capability. 
Ha learns something which can be generalized 
to other problems having similar formal char- 



acterlstics* ^Thls means he has acquired a new 
rule, or perhaps a new set of rules. (1974: 45) 

Although Bruner (1970) would hot agree w^ith Gagne that a new 
rule or even a now set of rules had necessarily been acquired in 
such a situation, he agrees that the cognitive strategies acquired 
in learning problem solving can be used to solve new problems-* that 
is, they exhibit transfer. Bruner points out the dif'f iculty in 
applying "conventional learning theories" to problem- solving behaviors: 
Rather, what seems to be at work in a good 
problem- solving 'performance' is some under- 
lying competence in using the operations df 
physics or whatever, and thob performance that 
emerges from this competence may never be the 
tame on any two occasions. What is learned 
it competence, not particular performances. 
(X970: 67) 

Gagne' s hierarchy,^ in which lower-order intellectual skills 
act as "sufficient conditions" for acquisition of higher-order skills, 
does not appear to apply as appropriately f6r problem- solving behaviors. 
The hierarchy, as it is employed in task analysis, deals with the , 
structure of the content to be taught. Problem-solving behaviors, 
however, depend on the learner's application of previously learned 
content, and no new content is actually taught. Thus the hierarchical 
task analysis appears to be inappropriate in design of instruction for 
problem solving. Rather than analyzing the sequence of prerequisite 
■kills and designing instruction to provide these skills as "sufficient 



conditions" for acquiring problei..- solving skills, the designer assumes 
tliie lower-level skills as entry skills and determines hc3W to provide 
for the learner the most ••favorable conditions'^ under which to prac- 
tice problem solving. Simulation appears to be an instructional tech- 
nique which can provide these necessary ••favorable conditions. •• 



Simulation in Learning 

As a strategy for teaching, eliciting and evaluating higher- 
level problem- solving skills, computer-based simulation has gained 
considerable attention recently. Simulation as a technique provides 
richness of stimuli, complex interrelationships between variables, 
and multiple possible responses and outcomes. The computer provides 
a viable means for managing this complexity. 

In the simulation setting, the learner is part of a dynamic, ^ 
self-regulating system. Human information processing in this context 
relies on continuous evaluation or monitoring of behavior (including 
thinking behavior) with consequent adjustments. -Miller, Galanter, 6 
Pribram (1960) have conceptualized this type of information processing 

a feedback loop they call a TOTE (for test-operate- test-exit ) . 
Simulation provides flexibility in the learning situation so that the 
learner can use feedback to test a thinking process or an action, 
modify it, test again, and continue in the loop until an objective is 

reached. _^ ' 

f 

Simulation is reality simplified. While students are learning 

V 0 

to identify the relevant variables in a situation and to combine those 
variables to achieve an optimal solution, they are exposed to sufficient 
rielliii«tt to ptovide credibility. At the same time they are protected 
ERICfifai fOM o£ the "noiie" that exists in the real decision-making letting. 
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Bobula t Page (1973) "describe the essence of simulation as 
, , , placing an individual in -a realistic 
•etting where he is confronted by a proble- 
matic situation that requires a sequence bf 
inquiries, decisions and actions. Each of 
these activities triggers appropriate feed- 
back which may modify the situation and be 
UiiBd for subsequent- decisions about what to 
do next. The examinee's next action in turn 
nay further modify the problem. Thus a pro- 
blem evolves through many stages "untiL it is 
terminated when the individual reaches an 
•cceptable solution or is f acied by unaccept- 
able, consequences brought about by his own 
choices and actions* (1972(: 1-1) 

I 

The individual in the simulation setting practices and refines 
•kills and strategies for recognizing, identifying and resolving 
problem situations. The cognitive strategies or competencies thus 
acquired should generalize, or transfer, to ottier similar problems: 
other simulations, or similar situations in real life. 

McGuire (1973) sees simulation as superior, not only to conven- 
tional methodology for teaching and testing, but also to reality. 
The advantages she suggests include perceived relevance, predetermina 
tion and preselection of the task (as opposed to reality) , standard- 
isation of the task (for testing), improved sampling of ^performance, 

I 

r o. 
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improved rating of performance , and increased responsibility and real- 
istic fieedback in a practical time frame. i 

Simulation is a technique vfhich can provide the ••favorable condi- 
tions**— the necessary realism, richness and complexity — for teaching 
and eliciting problem- solving behavior. More specifically, computer- 
based simulation can provide a flexible and individually responsive 
dimension, while automatically scoring and maintaining detailed records 
of each student's performance. Computer simulation, then, appears to 
provide a strategy and a medium uniquely sjuiited to instruction in 
problem- solving for large numbers of students* 

\ 

' Evaluation \ 

\ 

In addition to providing practice in problem-solving, a simulation 

of a patient management problem, a learning disability diagnostic problem,^ 

\ 

or a pre-trial legal procedures problem may be used to evaluate a students* 
problem- solving ability. In this instance, it is necessary to somehow, 
•core the •*^oodness|^ of the student's strategies* An objective evalua- 
tion requires that, somehow, problem- solving behavior be broken down 

into the skills demonstrated by a proficient problem- solver. Bobula and 

\ 

Page have suggested these generally applicable decision-making skills to 
be: 0 

1) discriminating between relevant and non-relevant issues 

2) recognizing the most important issues in the problem 

3) - determining what sequence to follow in order to solve t)}e 
^ problem i ^ 

4) obtaining aj^propriate and relevant ihformation^or data 

ERLC ^ . 9 . 
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5) interpreting data « 

6) avoiding unpecessary and wasteful actions (efficiency) 

7) manipulating a situation to alter it » 

8) monitoring the effects of this manipulation and taking 
corrective action in reaction to adve!rse effects 

9) applying substantive knowledge in solving the problem. 
10) resolving the problem most effectively (proficiency) by 

making the most appropriate decisions 



Before scores can be assigned' to the student's decision-making ^ 
skills/ criteria must be established by the subject matter expert and 
the ^instructional designer to judge the adequacy of each decision made 
by the student. These criteria reflect the complexity of the decision 
At each key decision point. "Correctness" af the option selected 
I dspends upon :khe conditions which exist at that point — conditions such 



I 



\ M information that has or has not been provided, decisions previously 

Suide or not made, and changes that have occurred in the problem setting 

in response to the student's decisions. Usually there is no single 

correct decision but a number of possible decisions varying in degree 

of correctness or appropriateness depending on existing conditions. 

These decision*making skills, which a student is to practice and 

■ iapii'oveiby using the simulation, characterize a general problem- solving 

I ■ . 

process, involving the application and interaction of these and other 

skills. ,Thus it is difficult to develop relevant, non-trivial criterion- 

referenced test items to measure the behaviors associated with the skills. 

• r 

» * V 
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Score Formulation: 

Errors of 1^% minus (the sum' ' 
Ommission of the positive points 
(t) chosen, divided by . 

maximum possible score) , 
converted. to percent. 



1 <3 



E.O.t « (100) - 



X 100 



Max. Score 



Example : 

For the above examinee: 
E.O. « 100 - 72 X 100 

E.O.^ - 20f 



Errors of 

Commission 

(%) 



The sum of the negative 
points, chosen, divided 
by maximum possible 
■core, converted^.to 
percent. • 



X 100 



Max. Score 



For the above examinee: 
" ^ X 100 

E.C.t - lit 

NCTE: 

100%-(E.O.t. E.Ct] - P% 
loot- [20% 4 lit] - 69 t 



Efficiency 
(t) 



Et 



The number of positively, 
weighted choices made, 
divided by the total 
nuipber of choices made,, 
converted to percent. 

No. of (•>•) choices 
No. of all choices ^ 



For the above examinees 

9 choices were 
, 2 choices were 0 i 

5 cl^oices were - 
15 



E 

.E 



if ^ f °° 
60t 



Competence 
Index (t) 



The Proficiency Xndex 
weighted by the 
Efficiency Index. ^ • 

(P x E/lOO) P . 
I 



For the above examinee: 
C 

' r 



(69 X 66/100) 69 .55. 
! 2 ^~ 



■MOTEt Any rational method of weighting proficiency can be 

employed to yield a competence score. The Competence . 
Index shown here is in use at the University of Illinois 
College of Medicine in scoring Simulated Problems in 
PS t lent Managemenib. < ^ , ^ 

12 ' - 
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NcGuire (1963? undated) and Levine & McGuire (1969) describe the 
use of simulations to evaluate such decision-making skii;.s in 'doctors. 
McGuire has developed exacting scoring procedures (Center for gduca-' 
tional Development, 1973) for these simulations. Her scoring techniques 
can be used effectively in various settings. o ' 

McGuire' s scoring scheme, as adojJted and modified by Bobula and 
Page (1973) yields five scores, or indices of performancie. The scores 
are. based on weights assigned to each oj>tion in the simulation. The 
weights may, for example, range from +16, +.8, +4, +2 for desirable 
actions to -2, -4, -8, or -16 for undesirable eictions. In some cases, 
failure to take a particularly crucial action may be scored as -16, 
whil« taking the action is scored as +16; the effective score for a 
person taking the action is thus +32. 

The scoring formtilas for evaluative simulations are defined as: 



Score 

Proficiency 
(%) 
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Formulation: 



The sum of (+) and (-) 
points for options 
chosen divided by^ the 
maximum possible score, 
converted to percent. 

P - I •*• (-)l X 100, 



Max.. Score 
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Example: 

Examinee X made the 
following choices on 
{ a written simulation 
where 90 was maximum 



score. 






No. of 






Choices 


Weight 


Sum 


3 


16 


"Iff 


2 


8 


16 


4 


2 


8 




0 


0 ' 


2 


.-1 


-2 




-4 


-8 


P - 1721 


+ l-ioi 


xlOO 




m 


s " 


P - 68.8 


or 69% 





When the formulas V€fre used to score performance on simulations 
in Civil Procedure (Edwards^ 1975)^ students' Proficiency and overall 
Competence scores improved significantly ^on a second simulation. At 
the same time^ however^ it is interesting to note that in many easels 
the Efficiency score decreased slightly. This is not surprising 
■tudents interacting with an initial simulation took far fewer actions, 
and those actions were likely to be obvious and correct. In dealing 
with a second simulation, students were aware of a much bibader set 
of possible actions, although a few of those turned out. to be inapprop- 
riate. ' 

student' s score on a simulation problem thieref ore depends on 
his/her thoroughness in choosing all those actions the expert, or 
practicing professionals, agree are important and discrimination in 
avoiding all those actions the expert^ agree are useless or harmful. 
The scores for Errors of Omission (failure to take desirable action) 
and Errors of Commission (taking an undesirable action) are helpful 
in analyzing the nature of the students' deficiencies. . 

Some Advantages and Limitations of Simulation as an Educati<5nal Tool 
Those who advocate the use of simulations for both teaching and 
tasting are sometimes challenged with the question, "If simulations 
of raality are so effective, why not use reality itself?" Assuming 

. it is possible to use reality (i.e., a real patient, a real client, ^ a . 

(- . ' • - 

real student) , ,there are powerful arguments against it. For many 
purposes, practitioners find simulation to be superior not only to ^ ' 




1 
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traditional methodology but also to reality itself. Simulation does 
not duplicate life, it imitates it. This may be, in the minds of some, 
its greatest limitation. To others, it is its greatest advantage. 

Often, an actual problem situation may be too complex for a student 
to deal with effectively. In a confused or chaotic real life se'tting 
it is difficult to segregate the important variables and see the rela- 
tionships between them. In a simulation, the problem situation may be 
highlighted by abstracting simple elements from the confusion of reality. 
The effects of interactions are intensified. A simulation can be complex 
conqpared to verbal models, but simple compared to the real world. This 
is a rewarding compromise: the simulation has enough complexity to 
account for the significant sources of variance in the real world, but 
is simple enough to^^be understood. 

A delicate balance must be achieved between realism and simplicity. 
He don't want to reproduce reality; rather, we want to isolate and 
highlight the important interrelationships in it. 

NO claim should be made that simulations are sufficient in and of 
themselves, or exhibit unqualified overall superiority over other media 
and techniques for instruction and evaluation. Clearly, some aspects 
of reality cannot at £he present time be economically simulated, if 
they can be simulated at all. In addition, it is important to realize 
. that simulation is not appropriate for teaching or for testing all 
aspects ot performance. For example, conventional objective tests are 
■till most economical and direct for measuring recall of factual inform- 
atlon. At tHb opposite extreme, professional effectiveness can only be 
eviiluated by observation over a long period of time in a diversity of 
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settings. Simulation can provide important advantages between these 
two extremes. Simulation can be, and should be, successfully combined 
and integrated with other instructional and evaluative approaches so 
that all students are reached by one or more techniques suited to the 
nature of the problem and their individual learning style. 
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